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@ A real time text translation system records audio/visual segments as well as a verbatim transcription 
of speech associated with the audio/visual segments and pointers that cross-index specific portions of 
the verbatim transcription with associated audio/visual segments. A retrieval system searches the 
previously stored verbatim transcription for specific words or phrases and then uses the associated 
pointers to allow rapid location of the audio/visual segments corresponding to the specific words or 
phrases located in the search. In this way, specific audio/visual segments can be located and retrieved 
quickly. 
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BACKGROUND OF THE INVENTION 

This invention relates to information storage systems that create a cross referenced file including an 
audio/visual recording of activities which include speech and a verbatim transcription of at least a portion of 

5 this speech, and to information retrieval systems that automatically search the verbatim transcription to locate 
desired portions of the audio/visual recording. 

At present there are many situations where users have a need to locate specific passages of an audio or 
visual recording. Typically, the user must search manually through the audio or visual recording to locate the 
desired passage. The time spent in such searching can be tedious and extended. 

10 For example, within the legal profession video recording is becoming more and more used as a litigation 

tool. Video recordings are being made of expert testimony, depositions and other evidence for later playback 
during discovery or in a court proceeding. When there is a need to locate a particular passage of a deposition, 
for example, a user will typically manually review video recordings to locate the desired segment. In a large 
case where there is a large volume of video recordings, this process can be so time consuming as almost to 

15 be impractical. 

Similarly, the broadcast industry also has a need to find specific prerecorded segments of an audio or visual 
recording. The industry has large inventories of prerecorded material, and this inventory is expected to grow 
at a rapid pace both in the near and long term. Given the large volume of audio and video recordings, it can 
be quite time consuming to locate a specific recorded segment. 

20 

SUMMARY OF THE INVENTION 

Accordingly, it is an object of this invention to provide an improved information storage system to allow rapid 
automatic retrieval of specific desired passages of an audio or visual recording. It is another object of this in- 
25 vention to provide an information retrieval system that will automatically locate desired portions of an audio or 
visual recording, or will automatically provide an address that will allow a user to manually locate such a seg- 
ment 

According to a first aspect of this invention, an information retrieval system is provided which includes 
means for automatically searching the entries of a composite record of the type comprising a series of 

30 audio/visual segments comprising speech at respective addresses on a recording medium, a verbatim tran- 
scription of at least a portion of said speech as a series of entries, and a series of pointers, each associated 
with a respective one of the entries and indicative of the address of an associated one of the audio/visual seg- 
ments to locate a first entry corresponding to a desired speech segment Once this first entry is located, the 
pointer associated with the entry can be used automatically to retrieve the associated audio/visual segment, 

35 or alternately the pointer can be automatically displayed to allow manual retrieval of the associated audio/visual 
segment. 

As described in detail below, a user can automatically search the verbatim transcription for a word or phrase 
of interest, and then use the system described below to play back the corresponding audio/visual segment auto- 
matically. In this way, the need for manual searching of the audio/visual record is substantially eliminated. 
40 According to a second aspect of this invention, an information storage system comprises means for pro- 

viding a series of audio/visual segments comprising speech at respective addresses on a recording medium 
and means for recording a three part composite file. Said recording means comprises means for storing a ver- 
batim transcription of at least a portion of the speech as a series of entries, each comprising at least one phone- 
tic outline and corresponding plain text derived from the phonetic outline, and means for storing a series of 
45 segment pointers, each associated with a respective one of the entries and indicative of the address of an as- 
sociated one of the audio/visual segments. 

This three part composite file synchronizes all three of the major parts of the recording: the phonetic outline, 
the plain text, and the audio/visual segments. 

The invention itself, together with further objects and attendant advantages, will best be understood by re- 
50 ference to the following detailed description, taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram of a presently preferred embodiment of the information storage system of this 
55 invention. 

Figure 2 is a flow chart of the synchronizing program executed by the computer of the system of Figure 1 . 

Figures 3a through 3g are flow charts of subroutines executed by the controller of Figure 2. 

Figure 4 is a block diagram of an information retrieval system which incorporates a presently preferred 
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embodiment of this invention. 

Figure 5 is a flow chart of the retrieval program executed by the computer of Figure 4. 
Figure 6 is a sample display generated by the system of Figure 4. 

5 DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS 

Turning now to the drawings, Figure 1 shows a block diagram of the hardware of the presently preferred 
embodiment 10 of the information storage system of this invention. The system 10 of Figure 1 includes a 
stenographic shorthand machine 12 connected to a computer 14. In the conventional way, the stenographic 

w shorthand machine 12 generates a series of outlines which are supplied to the computer 14 via a serial port 
included on the stenographic shorthand machine 1 2 and the serial port of the computer 1 4. As shown, the com- 
puter 14 includes a high resolution monitor 16 and a keyboard 18. 

Also as shown in Figure 1, a conventional video recorder 20 is connected to the computer 14 by means 
of a serial port and an RS-232 cable. This connection allows the computer 14 to supply commands to the video 

15 recorder 20 in order to control its operation, as well as to receive information from the video recorder 20, as 
described in greater detail below. The video recorder 20 is also connected to a video monitor or television 22 
as well as to video cameras 24 and microphones 26. The number of video cameras 24 and microphones 26 
can be varied to suit the application, and if desired video or audio switching equipment (not shown) can be pro- 
vided to allow selection of specific ones of the cameras 24 and microphones 26. 

20 The system 10 is intended for use for example in a courtroom or deposition setting where a court reporter 

uses the stenographic shorthand machine 12 to phonetically record the testimony of the proceedings, while 
the video recorder 20 records audio and visual signals from the microphones 26 and the cameras 24 to provide 
an audio/visual record of the proceedings. For example, each of the cameras 24 can be focused on a specific 
speaker, and the microphones 26 can be physically positioned to insure that the speech of each of the speakers 

25 is clearly recorded. 

Figure 2 provides a flow chart for the program executed by the computer 14, which will be referred to herein 
as the synchronizing program 30. This program 30 causes the computer 14 to create a synchronized record 
which includes stenographic outlines generated by the stenographic shorthand machine 12, plain text trans- 
lated from these outlines, as well as pointers which provide a cross-reference to associated portions of the 

30 audio/visual record made by the video recorder 20. 

As shown in Figure 2, the first step in the synchronizing program 30 is to power up the recorder 20 in block 
32, and then to check to insure that the recorder 20 has powered up properly. If so, the program 30 then checks 
in block 34 that a tape is properly installed in the recorder 20. If a fault is detected in either block 32 or 34, 
control passes to the controller 46 described below. The program 30 then sets the recording speed to be used 

35 by the recorder 20 and computes an end of tape address in block 36. The recorder 20 includes an internal in- 
dexing counter that maintains an address of the current video frame being recorded. The contents of this in- 
dexing counter can be read by the computer 14, and it is used in block 36 to compute the end of tape address 
at which the video tape in the recorder 20 will be exhausted. As explained below, this end of tape address is 
used to prompt the user to replace the tape before an end of tape condition is reached. The program 30 then 

40 obtains a tape label from the user in block 37 that will be used to identify the specific tape currently installed 
in the video recorder 20. In this embodiment the tape label is a four character field that is typed in by the user 
via the keyboard 18, and is then recorded on a tape control file maintained by the program 30, as well as on 
to the video recording tape by means of the recorder 20. 

The program 30 then commands the video recorder 20 in block 40 to begin recording on the tape, and in 

45 block 42 initializes a video address counter maintained by the computer 14. This is done by reading the internal 
indexing counter maintained by the video recorder 20 and setting the video address counter maintained by the 
computer 14 equal to this indexing counter. 

The computer 14 is programmed to increment the video address counter periodically at a rate that matches 
the recording rate of the video recorder 20 such that the video address counter maintained by the computer 

so 14 remains equal to the indexing counter of the video recorder 20. Additionally, the video address counter is 
periodically matched against the indexing counter of the video recorder 20 to insure a high degree of accuracy. 
If at any time the video address counter maintained by the computer 14 is found not to match the indexing count- 
er maintained by the video recorder 20, the video address counter of the computer 14 is reinitialized with the 
new and correct value. 

55 At this point, control is transferred via block 44 to a controller 46. As shown schematically in Figure 2, the 

controller 46 includes routines for handling a number of conditions which will be described below in detail in 
conjunction with Figures 3a through 3g. However, during normal operation the main task of the controller 46 
is to initiate reading of stenographic outlines of the stenographic shorthand machine 12 and then to translate 
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these outlines into text and to update the video address counter. These functions are performed in blocks 48 
and 50. Once this has been done, the program 30 writes a synchronized record in block 52 onto any suitable 
recording medium such as a magnetic disk. Reference numeral 54 schematically illustrates one of these 
synchronized records. As shown in Figure 2, each of the synchronized records includes the following six fields 
5 of information: 

(1) A stenographic byte count (SBC) which defines the number of bytes of recorded stenographic outlines 
(encompassing one or more stenographic entries); 

(2) The actual stenographic output made up of one or more phonetic outlines as written by the reporter on 
the stenographic shorthand machine 12; 

10 (3) A text byte count (TBC) representing the number of bytes of text translated from the stenographic out- 

lines of the synchronized record; 

(4) The translated text (so called "plain text") associated with the stenographic outlines of the synchronized 
record; 

(5) A video byte count (VBC) representing the number of bytes in the video address; and 
15 (6) The video address corresponding to the associated text of the synchronized record. 

The video address of each synchronized record is in this embodiment equal to the contents of the video 
address counter at a time corresponding to the associated stenographic outlines. Once the synchronized record 
has been stored, control is returned via block 44 to the controller 46 and the process is repeated. This process 
continues until the session is terminated as described below. 

20 The controller 46 additionally processes and handles various events during steno-to-text translation and 

text to audio/visual synchronization. The main events processed by the controller 46 will be described in con- 
junction with Figures 3a through 3g. Additional events such as conventional error condition handling routines 
are not described in detail. 

The end of tape (ET) routine is flow charted in Figure 3a. This routine is executed on a routine basis to 

25 alert the user of an impending end of tape condition. The routine computes an end of tape address (ETA) by 
interrogating the video recorder 20 and obtaining the video recording speed. This speed, when used with the 
maximum recording time for the recording tape in use, is then used in block 60 to compute the ETA. The ETA 
is then compared in block 62 with the current contents of the video address counter, and the routine branches 
in accordance with the results of this comparison. If an end of tape condition has occurred, the routine im- 

30 mediately branches at block 64 to the change tape (CT) routine of Figure 3. rf the video address counter is 
within five minutes of being equal to the ETA, the routine displays a message prompt in block 66 warning the 
user of an impending end of tape condition. This message prompt is displayed on the monitor 16 and control 
is then returned at block 68 to the controller 46. In the event the video address counter is less than the ETA 
by more than five minutes, no action is taken and control is returned to the controller 46 at block 70. 

35 The change tape (CT) routine of Figure 3b is executed when the recording tape has been completely re- 

corded on and yet the recording session is to be continued. The routine of Figure 3b first commands the recorder 
20 in block 72 to stop recording and then commands the recorder 20 in block 74 to eject the tape. The user is 
then prompted in block 76 to load a new tape and control is passed to the initiate recording (IR) routine of Figure 
3c. 

40 The initiate recording (IR) routine of Figure 3c first checks in block 78 to see if a tape has been mounted 

in the recorder 20. If not, the user is prompted in block 80 to mount a tape. Otherwise, the routine then sets or 
obtains a recording speed for the recorder 20 in block 82, obtains the tape label from the user in block 84, and 
then records the tape label both on the tape and on the tape control file in block 86. Control is then transferred 
to the resume recording (RR) routine of Figure 3g. 

45 The end of session (ES) routine of Figure 3d is executed in response to a request to stop audio/visual re- 

cording. The routine of Figure 3d first commands the recorder in block 88 to stop recording. It then terminates 
reading of outlines of the stenographic shorthand machine 12, terminates translation of stenographic outlines, 
and closes the synchronized file in block 90. The current value of the video address counter is then written in 
block 92 into the tape control file, and then the tape is ejected in block 94. Program execution is then terminated 

50 in block 96. 

The pause recording (PR) routine of Figure 3e is executed to cause the recorder 20 to pause recording. 
This command is issued in block 98. Then the video address counter maintained by the computer 14 is stopped 
while the recorder 20 is paused and control is transferred to the pause timer (PT) routine of Figure 3f. 

The pause timer (PT) routine of Figure 3f is used to control a pause condition lasting for a longer time than 
55 allowed by the video recorder 20. Most conventional video recorders have a maximum pause time. Once this 
time has been exceeded, the recorder automatically stops recording or resumes recording. In the event the 
user requires that the recording be paused longer than the maximum pause time, the routine of Figure 3f insures 
proper control of the recorder 20. The routine of Figure 3f starts by comparing in block 102 an elapsed pause 
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timer to determine whether or not it is greater than a maximum value MPT. No action is taken as long as the 
elapsed pause timer is less than MPT. However, once the elapsed pause timer exceeds MPT, the recorder 20 
is commanded to start recording for one second and the video address counter is reinitialized in block 104. 
Then the recorder 20 is commanded to pause recording and the video address counter is stopped in block 106. 
5 The routine then checks to determine whether the routine of Figure 3g has been requested by the user. If so, 
control branches to the routine of Figure 3g. Otherwise, control is returned to the beginning of the routine of 
Figure 3f. 

The resume recording (RR) routine of Figure 3g is executed to resume recording. This routine starts the 
recorder 20 in block 1 0B and initializes the video address counter of the computer 14 in block 110 prior to re- 

10 turning control to the controller 46. 

Once the synchronizing program 30 has been terminated with the ES routine of Figure 3d, the synchronized 
file can be used by a text processor to edit any untranslated stenographic outlines. Text may be added to the 
file without affecting the ability to search text which is associated with a synchronized video address. This feat- 
ure provides more flexibility for the user in text manipulation. Upon the completion of any required text editing, 

15 the synchronized file is ready to be read by the full text retrieval program described above. 

The information retrieval system 150 of Figure 4 can be used to locate specific text strings and to display 
the corresponding audio/visual segments. As shown in Figure 4, the retrieval system 150 includes a computer 
152 which includes a keyboard 154, a high resolution monitor 156, and a full motion video card 158. A video 
playback device 160 is connected to the full motion video card 158 via a serial port in the playback device 160 

20 and an RS-232 cable. The playback device 160 is connected to drive an audio speaker 162 and an optional 
video monitor 164. In the system 150 the audio/visual recording in the playback device 160 can be played back 
as a video image on the monitor 156 or on the monitor 164. The computer 152 in this preferred embodiment 
can be any PC-DOS personal computer. The playback device 160 may be the same device as the video re- 
corder 20, and in this preferred embodiment is preferably a video cassette recorder. 

25 Figure 5 is a flow chart of the retrieval program 1 70 executed by the computer 1 52. This program includes 

a conventional full text retrieval routine that operates to locate any selected text string within a file, and which 
has been modified to read the plain text entries of the synchronized files generated by the computer 14 as de- 
scribed above. This synchronized file contains the plain text to be searched, the video addresses associated 
with specific parts of the text required for video synchronization, an index to the text to facilitate search and 

30 retrieval functions, and a tape control file for tape labels and associated information. 

The retrieval program 170 of Figure 5 allows a user to search for any specific words or phrase directly or 
in conjunction with conventional boolean search criteria. The retrieval program is language independent, and 
it operates in block 172 to search a specific synchronized file for user requested text. When a match is found, 
the routine has automatically located the corresponding video segment, i.e., the video frame having a video 

35 address corresponding to the stored video address that is included in the synchronized file in association with 
the text located by the search of block 172. This is accomplished in block 174. The user may request to see 
the corresponding audio/visual segment by providing both beginning and ending addresses if the user wants 
a portion of the video record to be played and the playback device 160 paused at the end of the segment. 
Once playback has been requested, the routine checks in block 1 76 to insure that the tape header of the 

40 tape in the playback device 160 corresponds to the tape header for the synchronized file in use. In the event 
of a tape header mismatch, the'-user is prompted in block 178 to mount the correct tape. Once the tape header 
has been verified, the routine then displays in block 180 the text found in the search and the corresponding 
video addresses. The routine then issues a search command to the play back device 160 in block 182 for the 
video address corresponding to the text located in the search. The program logic will compare the current video 

45 address on the video tape to the requested video address from the synchronized file. If the requested video 
address is greater than the current video address retrieved from the video tape, a fast forward is initiated until 
the correct video address is located. If the requested video address is less than the current video address on 
the video tape, a rewind is requested until the correct video address is located on the video tape. When the 
requested video address is found on the video tape, the playback device 160 is commanded to playback the 

so video tape, and the resulting video images are displayed on the computer monitor 156, if the video card 158 
is in use, or on the optional video monitor 164 otherwise. At this point, the retrieval program 170 responds to 
other user commands in block 186, such as video and audio control commands issued from the computer 
keyboard 154 and other search and retrieval commands issued from the computer keyboard 154. 

The video card 158 allows the video image to be displayed in a number of formats on the monitor 156, as 

55 selected by the user, in this example, the user can command that the full screen of the monitor 156 be used 
to display the video image, or that 1/4 or 1/1 6 of the area of the screen of the monitor 156 be used for the video 
image. In the 1/4 or 1/16 screen mode, the video image may be moved around the screen of the video monitor 
(including completely or partially off of the display area). In the 1/4 or 1/16 screen mode, a partial view of the 
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video image can be shown with complete panning capabilities. Alternately, with the 1/4 screen mode and espe- 
cially with the 1/16 screen mode, the user may view the entire video image in a corner of the display of the 
monitor 156 while reading and manipulating text with the remainder of the monitor screen. For example, when 
the synchronized file and associated video tape relate to a court proceeding such as recorded testimony, the 

5 user can view the testimony and on 1/16 or 1/4 of the screen of the monitor 156, while the user follows along 
on the verbatim transcription of the synchronized record on the remainder of the screen. Similarly, other com- 
mands such as the initiation of other searches, retrieval the next text to be viewed, viewing new cases, ap- 
pending notes and changing to a new text database may be accomplished while the video playback is viewed 
on the monitor 156. As shown in block 188, the user may control the playback device 160 directly from the 

w keyboard 154 by issuing appropriate control commands including the following: pause, play, forward, fast for- 
ward, rewind, eject tape and other commands or control features supported by the playback device 1 60. If the 
user requires, video and audio playback may continue as other functions such as searches are performed by 
the system 150. 

Figure 6 shows by way of example the manner in which the verbatim transcription may be displayed on 

15 the monitor 156. Note that the display includes a header which includes the case name 200, the run date 202 
(the date the report was generated), the page number 204, the video address 206 corresponding to the first 
word in each line of text, the line number 208, and the actual text 210. As explained above, this text 210 is a 
verbatim transcription of the speech recorded on the associated video tape. 

The system 1 50 allows, for example, an attorney to use full text search techniques to search and find spo- 

20 ken plain text words or phrases used in a large textual database. Once the required words or phrase has been 
found, the user may then request to see or hear the corresponding portion of the audio/visual recording. The 
system provides tape header information for retrieving the correct tape, and will automatically search and find 
the corresponding audio/visual segment and then allow the user to playback the audio/visual segment corre- 
sponding to the text located in the search. The user may then control the video playback device for repeated 

25 viewing, or may initiate a new search. 

Similarly, the system 150 may be used in a audio/visual broadcast environment where there is a need to 
locate specific audio/visual segments that correspond to specific spoken words or phrases. The text may be 
in any spoken language that can be represented either now or in the future phonetically. With modem 
stenographic techniques, English, Spanish, German, French, Italian, Portuguese, Arabic and Korean can all 

30 be used with this invention. Of course, this list of languages is not meant to be complete or exhaustive. With 
the system 150, the user may use the full text search capabilities described above to search and locate specific 
requested portions of text. The text being searched may include comments or notes for use in finding specific 
audio/visual segments for ease of location or speaker identification. A broadcast user will have the ability to 
find the requested spoken phrase (regardless of language) and then to locate the corresponding audio/visual 

35 segment for display and playback purposes. The system 150 can be integrated into audio/visual editing 
software/hardware applications to facilitate the editing operation. 

The display provided by the retrieval program 170 can also be used by a user to facilitate manual control 
of a playback device to locate the recorded segment corresponding to a specific segment of text For example, 
the retrieval program 170 can be used to search automatically for the phrase "unfair market advantage" and 

40 to locate this text automatically at line 04 (Figure 6). The display indicates to the user that video address 00009 
corresponds to the video frame of the first word of line 04. The user can then use conventional manual controls 
on the playback device 160 to advance or rewind the tape to video address 00009 for playback. In this mode 
of operation, the routine has not automatically commanded playback of the associated portion of the videotape, 
but has still provided the user with the video address that allows this function to be performed quickly and ef- 

45 ficiently. 

Table 1 lists presently preferred components suitable for use in the storage system 10 and the retrieval 
system 150. Though the identified components have been found suitable, the invention is of course capable 
of being implemented on a wide variety of other hardware. 

50 
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TABLE 1 



Reference 


i No. 


Description 


Tdenfcif ication 


12 




S t enog r aphi c 
shorthand machine 


SmartWriter 
(Stenograph 
Corp . ) 




152 


Step 486/25 
personal computer 


Everex Model EXO 

2904C- 01 


16, 


156 


super VGA monitor 


Everex Model MON 

00500 - OA 


20 , 


160 


stereo video 
cassette recorder 


NEC Model PC -VCR 
PV-S98A 


22, 


164 


video monitor 


Sony Model KV- 
32XBR15 


24 




video -audio 
camera 


Sony Model No. 
CCD-V11 


158 




full motion video 
card 


Video Mail Model 
VMC-1E 


162 




amplified speaker 


Radio Shack 
Minimus - 0 - 8 



30 The video cassette recorder identified above supports a large set of commands, controls, status codes and 

error codes, and can be easily connected to a computer to allow the computer to control all of the functions 
and to obtain all of the status information described above. 

From the foregoing, it should be apparent that an improved information storage system has been described 
which includes a means for providing a series of audio/visual segments comprising speech at respective ad- 

35 dresses on a recording medium. In the illustrated example, this providing means includes the video recorder 
20 which actually records the audio/visual segments in real time as they occur. As explained above, this system 
is well suited, for example, for the recording of depositions or other ongoing events. In this case, a stenographic 
shorthand machine can be used to create the verbatim transcription in real time, substantially simultaneously 
with the recording of the audio/visual segments. 

40 However, it is not required in all embodiments of this invention that the audio/visual segments be recorded 

at the same time that the verbatim transcription is created. For example, this invention can be used with a pre- 
recorded video tape including both audio and visual segments. When used in this mode, the prerecorded video 
tape replayed and a stenographic shorthand machine used to create a verbatim transcription during playback 
rather than during original recording. In this alternative arrangement, a synchronized file is created which is 

45 quite similar to that described above, and which provides all the advantages described above in terms of high 
speed automatic retrieval of specific audio/visual segments. 

As explained above, the system 10 includes a means for recording a verbatim transcription of speech as 
a series of entries. In the preferred embodiment 10, this means includes the stenographic shorthand machine 
12 and associated software in the computer 14. In future it is anticipated that speech recognition systems will 

so be developed which may for some or all applications eliminate the need for a stenographic shorthand machine 
and a court reporter. As used herein, the term verbatim transcription is intended broadly enough to include com- 
puter generated verbatim transcriptions which are generated by means of voice recognition techniques. 

The system 10 also includes a means for automatically storing a series of pointers, each associated with 
specific verbatim transcription entry and indicative of the address of the associated audio/visual segment. In 

55 the preferred embodiment described above, these pointers correspond to the video addresses generated by 
the video recorder 20. However, in alternate embodiments other pointers may be used such as time signals or 
sequence signals of other types. 

In the system 10 the audio/visual segments are audio and video signals recorded by the recorder 20. It is 
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10 



30 



35 



not required in all embodiments of this invention that the audio/visual segments include any visual information 
at alt. Thus, this invention is well suited for use in connection audio tapes which include no visual information 
of any type. Of course, this invention is not limited to any particular medium for recording audio/visual segments. 
The preferred embodiments discussed above utilize video tape, but magnetic disks, CD ROMS, optical drives, 
or other recording media may be adapted for use with this invention. 

Of course, it is intended that the foregoing detailed description be regarded as illustrative rather than limi- 
ting, and that it be understood that it is the following claims, including all equivalents, which are intended to 
define the scope of this invention. 



Claims 



1. An information retrieval system comprising: 

means for storing a composite record comprising: 
15 a series of audio/visual segments comprising speech recorded at respective addresses on 

a recording medium; 

a verbatim plain text transcription of at least a portion of the speech recorded as a series of 

entries; 

a series of pointers, each associated with a respective one of the entries and indicative of 
20 the address of an associated one of the series of audio/visual segments; 

means for automatically searching the series of entries to locate a first entry corresponding to a 
desired speech segment in plain text; and 

means for automatically utilizing the pointer associated with the first entry to automatically retrieve 
the associated one of the series of audio/visual segments. 

25 

2. The invention of Claim 1 further comprising: 

means for automatically displaying the first entry and the associated one of the series of audio/visual 
segments together on a single display screen. 

3. The invention of Claim 1 wherein the information retrieval system comprises a computer and means for 
playing back the series of-audio/visual segments; wherein the searching means and the utilizing means 
are included in the computer; and wherein the computer comprises means for controlling operation of the 
means for playing back the series of audio/visual segments. 

4. The invention of Claim 1 wherein the recording medium comprises a machine readable label, and wherein 
the invention further comprises means for automatically verifying that the label corresponds to the pointer 
associated with the first entry. 

5. An information retrieval system comprising: 

means for storing a composite record comprising: 
40 a series of audio/visual segments comprising speech recorded at respective addresses on 

a recording medium; 

a verbatim transcription of at least a portion of the speech recorded as a series of entries; 

a series of pointers, each associated with a respective one of the entries and indicative of 
the address of an associated one of the series of audio/visual segments; 
45 means for automatically displaying the pointer associated with the first entry to allow retrieval 

of the associated one of the series of audio/visual segments. 

6. The invention of Claim 5 wherein the displaying means comprises means for automatically displaying the 
pointer associated with the first entry and the first entry together on a single display screen. 

50 

7. An information storage system comprising: 

means for providing a series of audio/visual segments comprising speech at respective addresses 
on a recording medium; 

means for recording a three part composite file, said recording means comprising: 
55 means for recording a verbatim transcription of at least a portion of said speech as a series of en- 

tries, each of said entries comprising at least one phonetic outline and corresponding plain text derived 
from the at least one phonetic outline; and 

means for automatically storing a series of segment pointers, each of said pointers associated with 
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a respective one of said entries and indicative of the address of an associated one of the series of 
audio/visual segments. 

8. The invention of Claim 7 wherein the transcription recording means comprises a stenographic machine 
operative to generate a series of outlines, and means for storing the series of outlines. 

9. The invention of Claim 7 wherein the providing means comprises means for recording a visual record of 
said series of audio/visual segments in real time. 

10. The invention of Claim 7 wherein the providing means comprises means for providing a said series of 
audio/visual segments as a prerecorded record. 
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Figure 1 
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Figures 
3a -3g 
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Figure 4 
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Figure 6 
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